Combining online and offline recognizers in a handwriting recognition system

ABSTRACT

Described is a technology by which online recognition of handwritten input data is combined with offline recognition and processing to obtain a combined recognition result. In general, the combination improves overall recognition accuracy. In one aspect, online and offline recognition is separately performed to obtain online and offline character-level recognition scores for candidates (hypotheses). A statistical analysis-based combination algorithm, an AdaBoost algorithm, and/or a neural network-based combination may determine a combination function to combine the scores to produce a result set of one or more results. Online and offline radical-level recognition may be performed. For example, a HMM recognizer may generate online radical scores used to build a radical graph, which is then rescored using the offline radical recognition scores. Paths in the rescored graph are then searched to provide the combined recognition result, e.g., corresponding to the path with the highest score.

This application is a continuation of, and claims priority from, U.S.application Ser. No. 11/823,644, filed Jun. 28, 2007, which isincorporated herein by reference in its entirety.

BACKGROUND

To recognize a handwritten input character, various types of recognitionmodels may be applied for classification purposes, such as an onlinerecognition model (e.g., a Hidden Markov Model) or an offlinerecognition model (e.g., a statistical template-based model).

However, different error sets result from different types of recognitionmodels. As a result, while both types of recognition models provide verygood classification performance, the models have different error caseson a given dataset and thus the recognition accuracy suffers to anextent depending on the dataset.

SUMMARY

This Summary is provided to introduce a selection of representativeconcepts in a simplified form that are further described below in theDetailed Description. This Summary is not intended to identify keyfeatures or essential features of the claimed subject matter, nor is itintended to be used in any way that would limit the scope of the claimedsubject matter.

Briefly, various aspects of the subject matter described herein aredirected towards a technology by which online recognition of handwritteninput data is combined with offline recognition, to obtain a combinedrecognition result. In general, the combination improves overallrecognition accuracy.

In one aspect, online recognition and offline recognition are separatelyperformed to obtain character-level online and offline recognitionresult sets. The online and offline recognition result sets are combinedto obtain the combined recognition result. For example, the onlinerecognizer produces online hypotheses, each having a score; the offlinerecognizer produces offline hypotheses, each having a score. Astatistical analysis-based combination combines the scores to determinesimilarities to the handwritten input. Alternatively, (or in additionto), the online and offline scores for the handwritten input may beconsidered as features, to which an AdaBoost algorithm is applied toproduce a combination function in feature space composed of online andoffline scores to combine the online scores with the offline scores.Alternatively, (or in addition to), the online and offline scores arecombined using neural network-based combination, e.g., by applying aback propagation algorithm.

In one aspect, combining online recognition with offline processingcomprises performing online recognition to obtain radical level onlinerecognition data, which is then used in a radical graph. Offlinerecognition processing uses radical level offline recognition data onthe online recognition data to obtain the combined recognition result.For example, a HMM recognizer may generate a radical graph that anoffline process processes by rescoring the radical graph with offlineradical level data to obtain the combined recognition result.

Other advantages may become apparent from the following detaileddescription when taken in conjunction with the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention is illustrated by way of example and not limitedin the accompanying figures in which like reference numerals indicatesimilar elements and in which:

FIG. 1 shows an illustrative block diagram example of a system forcombining offline and online recognition results using statisticalanalysis based combination, AdaBoost-based combination, or neuralnetwork-based combination.

FIG. 2 is a representation of a neural network system, in whichBack-Propagated Delta Rule Networks (BP) may be applied to combineonline and offline recognition models.

FIG. 3 shows an illustrative block diagram example of a system forcombining offline and online recognition results using graph-basedcombination.

FIG. 4 is a representation of a simplified radical graph that may begenerated by online recognition and rescored by offline recognition.

FIG. 5 is a flow diagram representing example steps taken to combineonline and offline recognition models using statistical analysis basedcombination, AdaBoost-based combination, or neural network-basedcombination.

FIG. 6 is a flow diagram representing example steps taken to combineonline and offline recognition models using graph-based combination.

FIG. 7 shows an illustrative example of a general-purpose networkcomputing environment into which various aspects of the presentinvention may be incorporated.

DETAILED DESCRIPTION

Various aspects of the technology described herein are generallydirected towards improving handwritten input (including one or morecharacters, symbols, gestures, shapes, equations and so forth)recognition performance by combining offline and online recognitionmodels, particularly (but not necessarily) for use in recognizingEastern Asian (EA) characters. In one set of examples, to achieve higherrecognition accuracy, a Hidden Markov Model was used as the onlinerecognition model, and was combined with an offline recognition modelcomprising statistical analysis-based model, an AdaBoost-based model, aneural network-based model, and/or a graph based model.

As will be understood, however, these are only example models that maybe combined, and other models are similarly combinable. As such, thepresent invention is not limited to any particular embodiments, aspects,concepts, structures, functionalities or examples described herein.Rather, any of the embodiments, aspects, concepts, structures,functionalities or examples described herein are non-limiting, and thepresent invention may be used various ways that provide benefits andadvantages in computing and character recognition technology in general.

FIG. 1 shows the general concept of one example type of combinedrecognition system 100, in which an online recognition model(recognizer) 102 and an offline recognition model (recognizer) 104 arecombined to achieve better recognition (classification) performance. Ingeneral, given an input character 106, the online recognizer 102recognizes the input as an online recognition result 108 separately fromthe offline recognizer 104, which provides its own offline recognitionresult 110.

As set forth below, one or more of various combiner algorithms are usedas a combiner mechanism 112 to combine the recognition results 108 and110 into a final recognition result (classification) 114. Note that therecognition results need not necessarily be in the form of a recognizedcharacter, but may include various scoring and other data (features) ofthe input character that may be used during the combination process.

For example, for an input character

the recognition results are as follows:

Offline results are set forth in the following table, in which thesmaller the score, the more similar the input character is to therecognition hypothesis:

hypotheses scores Top1

299.776825 Top2

360.055176 Top3

395.169220 Top4

395.681610 Top5

409.075226 Top6

415.944855 Top7

417.508453 Top8

426.750671 Top9

427.738159 Top10

431.765106

Online results are set forth in the following table, in which the higherthe score is, the more similar the input character is to the recognitionhypothesis:

hypotheses scores Top1

−6.887748 Top2

−7.297907 Top3

−7.433957 Top4

−7.599403 Top5

−7.630971 Top6

−7.700678 Top7

−7.730252 Top8

−7.891527 Top9

−7.921412 Top10

−8.128083

If S is the recognition score, the score normalization S_(n) is definedas follows:

$S_{n} = \frac{S - S_{\max}}{S_{\max} - S_{\min}}$

where S_(min) is the minimal score, S_(max) is the maximal score in therecognition score results.

The following table shows the combined results using a statisticalanalysis-based combination method (product rule); the smaller the score,the more similar the input character is to the recognition hypothesis:

hypotheses scores Top1

0.316077 Top2

0.382694 Top3

0.579777 Top4

0.592984 Top5

0.595682 Top6

0.605659 Top7

0.609075 Top8

0.615945 Top9

0.617509 Top10

0.626751

For example, as set forth below, a score may be provided for the N-besthypotheses. Then the online and offline scores are combined by analgorithm, with the recognition result coming out of the besthypothesis.

A first such example algorithm used as the combiner mechanism comprisesa statistical analysis-based combination process. A statisticalanalysis-based combination process is set forth below:

Suppose S₁ is the normalized score of online recognition and S₂ is thenormalized score of offline recognition. Fuzzy features are used torepresent S₁ and S₂. Each character is associated with a fuzzy featurethat assigns a value (between 0 and 1) to each feature vector in thefeature space. A fuzzy feature {tilde over (F)} on the feature space

is defined by a mapping μ_({tilde over (F)}):

→[0,1] named as the membership function.

For any feature vector {right arrow over (f)}∈

the value of μ_(F)({right arrow over (f)}) is called the degree ofmembership of {right arrow over (f)} to the fuzzy feature {tilde over(F)}. When the value of μ_(F)({right arrow over (f)}) is closer to 1,the input character is more similar to the template character. For thefuzzy feature {tilde over (F)}, there is a smooth transition for thedegree of membership to {tilde over (F)} besides the hard cases {rightarrow over (f)}∈{tilde over (F)}(μ_({tilde over (F)})({right arrow over(f)})=1) and {right arrow over (f)}∉{tilde over(F)}(μ_({tilde over (F)})({right arrow over (f)})=0). A fuzzy featuredegenerates to a conventional feature set if the range of μ_(F) is {0,1}instead of [0,1].

Building or choosing a proper membership function is anapplication-dependent issue; commonly-used membership functions arecone, exponential, and Cauchy functions. In one example implementation,the Cauchy function is used due to its good expressiveness andhigh-computational efficiency.

The Cauchy function:C:

→[0,1], is defined as:

${C\left( \overset{\rightarrow}{x} \right)} = \frac{1}{1 + \left( \frac{{\overset{\rightarrow}{x} - \overset{\rightarrow}{v}}}{d} \right)^{\alpha}}$

where {right arrow over (v)}∈

d and a∈

d>0, a>=0. In this function, {right arrow over (v)} is the centerlocation of the fuzzy set, d represents the width (∥{right arrow over(x)}−{right arrow over (v)}∥ for C({right arrow over (x)})=0.5) of thefunction, and α determines the smoothness of the function. Generally, dand a portray the grade of fuzziness of the corresponding fuzzy feature.For fixed d, the grade of fuzziness increases as α decreases. For fixedα, the grade of fuzziness increases as d increases.

Accordingly, feature S₁ is represented by a fuzzy feature whosemembership function, μ_(S) ₁ :

→[0,1], is defined as:

${\mu_{S_{1}}:\left. \rightarrow\left\lbrack {0,1} \right\rbrack \right.} = \left\{ \begin{matrix}\frac{1}{1 + \left( \frac{{S_{1} - S_{c\; 1}}}{d_{1}} \right)^{\alpha_{1}}} & {{{S_{1} - S_{c\; 1}}} < {Thre}} \\0 & {otherwise}\end{matrix} \right.$

where S_(c1) is cluster center of fuzzy feature set, {tilde over (S)}₁,∥S₁−S_(c1)∥ represents the distance between feature S₁ and S_(c1), andThre is an empirical parameter.

The feature S₂ is represented by fuzzy feature whose membershipfunction, μ_(S) ₂

→[0,1], is defined as:

${\mu_{S_{2}}:\left. \rightarrow\left\lbrack {0,1} \right\rbrack \right.} = \left\{ \begin{matrix}\frac{1}{1 + \left( \frac{{S_{2} - S_{c\; 2}}}{d_{2}} \right)^{\alpha_{2}}} & {{{S_{2} - S_{c\; 2}}} < {Thre}} \\0 & {otherwise}\end{matrix} \right.$

where S_(c2) is cluster center of fuzzy feature set {tilde over (S)}₂,and ∥S₂−S_(c2)∥ represents the distance between feature S₂ and S_(c2).

An intrinsic property of such membership functions is that the farther afeature vector moves away from the cluster center, the lower the degreeof membership is to the fuzzy feature. At the same time, the degrees ofmembership to the other fuzzy features may be increasing. This describesthe gradual transition of two clusters.

A product rules and/or a summation rule may be used to combine μ_(S) ₁and μ_(S) ₂ . In these rules, the similarity is in the real interval[0,1] because μ_(S) ₁ and μ_(S) ₂ always within [0,1].

The following sets forth the product rule:

Similarity=μ_(S) ₁ ^(p)*μ_(S) ₂ ^(1-p)

The following sets forth the summation rule:

Similarity=p*μ _(S) ₁ +(1−p)*μ_(S) ₂

Turning to another process, an AdaBoost-based combination process may beused in EA Recognition. In general, the AdaBoost algorithm solves manypractical difficulties of earlier boosting algorithms. AdaBoost calls agiven weak or base learning algorithm repeatedly in a series of roundst=1 . . . T. One of the main ideas of the algorithm is to maintain adistribution or set of weights over the training set. The weight of thisdistribution on training example i on round t is denoted Dt(i).Initially, all weights are set equally, but on each round, the weightsof incorrectly classified examples are increased so that the weaklearner is forced to focus on the hard examples in the training set.

AdaBoost is well-known algorithm in the machine learning field, andsolves questions related to classification. Herein is described thesolving of combination problems using AdaBoost.

In general, online and offline scores may be considered as features,with AdaBoost applied to this feature pool, to get T weak classifiersh_(t)(s_(ti)),t=1 . . . T,i=1 or 2. The final combination result isoutput as

${{H\left( {s_{1},s_{2}} \right)} = {\sum\limits_{t = 1}^{T}{h_{t}\left( s_{ti} \right)}}},{s_{ti} = {1\mspace{14mu} {or}\mspace{14mu} 2.}}$

In an alternative, neural network system, Back-Propagated Delta RuleNetworks (BP) was also applied to combine online and offline recognitionmodels. BP networks developed from the simple Delta rule in which extrahidden layers (layers additional to the input and output layers, notconnected externally) are added. The network topology is constrained tobe feedforward, that is, loop-free; generally connections are allowedfrom the input layer to the first (and possibly only) hidden layer; fromthe first hidden layer to the second, and so forth, from the last hiddenlayer to the output layer.

FIG. 2 shows an example back propagation network 250. A first layer ofthe network has two nodes that represent the input; s₁ and s₂respectively represent the offline and online recognition score.

A sigmoid function ƒ₁(x) and ƒ₂(x) is applied to the offline and onlinescores so that ƒ₁(s₁) and ƒ₂(s₂) ranges between 0 and 1.

${{f_{1}(x)} = \frac{1}{1 - ^{- {\lambda_{1}{({x - \theta_{1}})}}}}},{{f_{2}(x)} = {\frac{1}{1 + ^{- {\lambda_{2}{({x - \theta_{2}})}}}}.}}$

As also shown, ƒ₁(s₁) and ƒ₂(s₂) are the outputs of a second layer. Athird layer outputs the linear combination result of the second layer'soutputs.

The network's final output is computed as:

$o = {\frac{1}{1 + ^{- {net}}}.}$

The back propagation algorithm employs gradient descent to learn theweights ω₁,ω₂ and parameters λ₁,λ₂,θ₁,θ₂ of this multilayer network.

An alternative graph-based combination system 300 is represented inFIGS. 3 and 4. Note that unlike the combination system of FIG. 1, inwhich the online and offline recognition processes are separatelyperformed and then combined, in graph-based combination, an onlinerecognition process 302 is first performed on an input 306 to produce aresultant graph 310, which is then processed by an offline rescoringmechanism 304 into a final recognition result 314.

In one example implementation, for a written input character 306, theonline HMM model is used as the recognizer 310, which decodes andgenerates a radical graph 310 containing multiple hypotheses anddetailed score information. A simplified example of a generated radicalgraph 410 is shown in FIG. 4, where a radical is a common component ofat least two characters.

The radical graph 310 is an intermediate result of HMM recognition. TheHMM recognizer tries to separate ink of a character into several partsand recognize each part to a possible radical. In the radical graph,each edge is a recognized radical and its score of possibility.

After the radical graph 310 is obtained, each path from START to ENDmeans a character which comprises several sequential radicals in thepath. A whole character model searches for a path in the graph havingthe highest score. The character represented by the path will be finalrecognition result, (or if multiple results are returned, they areranked by their path scores).

Based on the radical graph 310, the offline rescoring model 304 rescoresthe graph 310 into a rescored radical graph 320. To this end, theoffline rescoring model 304 includes a radical based recognition model322, that uses feature data 330 including radical-based relationfeatures 331, position features 332, duration features 333 and/or otherfeatures 334. Once rescored, the best hypothesis (or hypotheses ifmultiple results are desired) can be selected from the graph by thescore via an optimal path searching mechanism 340.

Turning to an explanation of the operation of the various components,FIG. 5 represents performing online recognition (step 502) and offlinerecognition (step 504) separately, to provide online and offlinecharacter-level recognition results, and then combining the results(step 506) using the combiner mechanism 112 of FIG. 1. As is understood,steps 502 and 504 may be performed in any order, and/or at least some ofthe recognition processing represented thereby may be performed inparallel.

As represented in step 506, the combiner mechanism combines the onlineand offline results, and may use statistical analysis based combination,AdaBoost-based combination, or neural network-based combination on theresults. Which combination type is used may depend on many factors,e.g., the processing power of the device performing the combination.

The final recognition result set is then output as represented by step508, which may be a single best result or a group of ranked results.Note that it is also feasible to perform more than one such type ofcombination, and then output a result set based on a confidence level(e.g., normalized as necessary) for each combination result.

FIG. 6 represents the radical level recognition, including radicalgraph-based combination, in which online recognition (e.g., HMM) isperformed as represented by step 602 to output a radical graph from theonline results. An offline process (step 604) uses offline radical-levelresults to subsequently process the interim online result, e.g., rescorethe radical graph as described above.

As represented in step 606, a final recognition result is then output,which may be a single best result or a set of score-ranked results. Notethat it ds also feasible to perform any or all of the combination typesof FIG. 5 and combine the results therefrom with the results of thecombined graph-based online and offline recognition model of FIG. 6, andthen output a result set based on a confidence level (e.g., normalizedas necessary) for each combination result.

Exemplary Operating Environment

FIG. 7 illustrates an example of a suitable computing system environment700 on which the recognition system 100 of FIG. 1 and/or the recognitionsystem 300 of FIG. 3 may be implemented. The computing systemenvironment 700 is only one example of a suitable computing environmentand is not intended to suggest any limitation as to the scope of use orfunctionality of the invention. Neither should the computing environment700 be interpreted as having any dependency or requirement relating toany one or combination of components illustrated in the exemplaryoperating environment 700.

The invention is operational with numerous other general purpose orspecial purpose computing system environments or configurations.Examples of well known computing systems, environments, and/orconfigurations that may be suitable for use with the invention include,but are not limited to: personal computers, server computers, hand-heldor laptop devices, tablet devices, multiprocessor systems,microprocessor-based systems, set top boxes, programmable consumerelectronics, network PCs, minicomputers, mainframe computers,distributed computing environments that include any of the above systemsor devices, and the like.

The invention may be described in the general context ofcomputer-executable instructions, such as program modules, beingexecuted by a computer. Generally, program modules include routines,programs, objects, components, data structures, and so forth, whichperform particular tasks or implement particular abstract data types.The invention may also be practiced in distributed computingenvironments where tasks are performed by remote processing devices thatare linked through a communications network. In a distributed computingenvironment, program modules may be located in local and/or remotecomputer storage media including memory storage devices.

With reference to FIG. 7, an exemplary system for implementing variousaspects of the invention may include a general purpose computing devicein the form of a computer 710. Components of the computer 710 mayinclude, but are not limited to, a processing unit 720, a system memory730, and a system bus 721 that couples various system componentsincluding the system memory to the processing unit 720. The system bus721 may be any of several types of bus structures including a memory busor memory controller, a peripheral bus, and a local bus using any of avariety of bus architectures. By way of example, and not limitation,such architectures include Industry Standard Architecture (ISA) bus,Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, VideoElectronics Standards Association (VESA) local bus, and PeripheralComponent Interconnect (PCI) bus also known as Mezzanine bus.

The computer 710 typically includes a variety of computer-readablemedia. Computer-readable media can be any available media that can beaccessed by the computer 710 and includes both volatile and nonvolatilemedia, and removable and non-removable media. By way of example, and notlimitation, computer-readable media may comprise computer storage mediaand communication media. Computer storage media includes volatile andnonvolatile, removable and non-removable media implemented in any methodor technology for storage of information such as computer-readableinstructions, data structures, program modules or other data. Computerstorage media includes, but is not limited to, RAM, ROM, EEPROM, flashmemory or other memory technology, CD-ROM, digital versatile disks (DVD)or other optical disk storage, magnetic cassettes, magnetic tape,magnetic disk storage or other magnetic storage devices, or any othermedium which can be used to store the desired information and which canaccessed by the computer 710. Communication media typically embodiescomputer-readable instructions, data structures, program modules orother data in a modulated data signal such as a carrier wave or othertransport mechanism and includes any information delivery media. Theterm “modulated data signal” means a signal that has one or more of itscharacteristics set or changed in such a manner as to encode informationin the signal. By way of example, and not limitation, communicationmedia includes wired media such as a wired network or direct-wiredconnection, and wireless media such as acoustic, RF, infrared and otherwireless media. Combinations of the any of the above should also beincluded within the scope of computer-readable media.

The system memory 730 includes computer storage media in the form ofvolatile and/or nonvolatile memory such as read only memory (ROM) 731and random access memory (RAM) 732. A basic input/output system 733(BIOS), containing the basic routines that help to transfer informationbetween elements within computer 710, such as during start-up, istypically stored in ROM 731. RAM 732 typically contains data and/orprogram modules that are immediately accessible to and/or presentlybeing operated on by processing unit 720. By way of example, and notlimitation, FIG. 7 illustrates operating system 734, applicationprograms 735, other program modules 736 and program data 737.

The computer 710 may also include other removable/non-removable,volatile/nonvolatile computer storage media. By way of example only,FIG. 7 illustrates a hard disk drive 741 that reads from or writes tonon-removable, nonvolatile magnetic media, a magnetic disk drive 751that reads from or writes to a removable, nonvolatile magnetic disk 752,and an optical disk drive 755 that reads from or writes to a removable,nonvolatile optical disk 756 such as a CD ROM or other optical media.Other removable/non-removable, volatile/nonvolatile computer storagemedia that can be used in the exemplary operating environment include,but are not limited to, magnetic tape cassettes, flash memory cards,digital versatile disks, digital video tape, solid state RAM, solidstate ROM, and the like. The hard disk drive 741 is typically connectedto the system bus 721 through a non-removable memory interface such asinterface 740, and magnetic disk drive 751 and optical disk drive 755are typically connected to the system bus 721 by a removable memoryinterface, such as interface 750.

The drives and their associated computer storage media, described aboveand illustrated in FIG. 7, provide storage of computer-readableinstructions, data structures, program modules and other data for thecomputer 710. In FIG. 7, for example, hard disk drive 741 is illustratedas storing operating system 744, application programs 745, other programmodules 746 and program data 747. Note that these components can eitherbe the same as or different from operating system 734, applicationprograms 735, other program modules 736, and program data 737. Operatingsystem 744, application programs 745, other program modules 746, andprogram data 747 are given different numbers herein to illustrate that,at a minimum, they are different copies. The term “computer storagemedia” as used herein, distinct from “communications media”, refers tostatutory articles of manufacture configured for storingcomputer-executable instructions and that are not signals or carrierwaves per se.

A user may enter commands and information into the computer 710 throughinput devices such as a tablet, or electronic digitizer, 764, amicrophone 763, a keyboard 762 and pointing device 761, commonlyreferred to as mouse, trackball or touch pad. Other input devices notshown in FIG. 7 may include a joystick, game pad, satellite dish,scanner, or the like. These and other input devices are often connectedto the processing unit 720 through a user input interface 760 that iscoupled to the system bus, but may be connected by other interface andbus structures, such as a parallel port, game port or a universal serialbus (USB). A monitor 791 or other type of display device is alsoconnected to the system bus 721 via an interface, such as a videointerface 790. The monitor 791 may also be integrated with atouch-screen panel or the like. Note that the monitor and/or touchscreen panel can be physically coupled to a housing in which thecomputing device 710 is incorporated, such as in a tablet-type personalcomputer. In addition, computers such as the computing device 710 mayalso include other peripheral output devices such as speakers 795 andprinter 796, which may be connected through an output peripheralinterface 794 or the like.

The computer 710 may operate in a networked environment using logicalconnections to one or more remote computers, such as a remote computer780. The remote computer 780 may be a personal computer, a server, arouter, a network PC, a peer device or other common network node, andtypically includes many or all of the elements described above relativeto the computer 710, although only a memory storage device 781 has beenillustrated in FIG. 7. The logical connections depicted in FIG. 7include one or more local area networks (LAN) 771 and one or more widearea networks (WAN) 773, but may also include other networks. Suchnetworking environments are commonplace in offices, enterprise-widecomputer networks, intranets and the Internet.

When used in a LAN networking environment, the computer 710 is connectedto the LAN 771 through a network interface or adapter 770. When used ina WAN networking environment, the computer 710 typically includes amodem 772 or other means for establishing communications over the WAN773, such as the Internet. The modem 772, which may be internal orexternal, may be connected to the system bus 721 via the user inputinterface 760 or other appropriate mechanism. A wireless networkingcomponent 774 such as comprising an interface and antenna may be coupledthrough a suitable device such as an access point or peer computer to aWAN or LAN. In a networked environment, program modules depictedrelative to the computer 710, or portions thereof, may be stored in theremote memory storage device. By way of example, and not limitation,FIG. 7 illustrates remote application programs 785 as residing on memorydevice 781. It may be appreciated that the network connections shown areexemplary and other means of establishing a communications link betweenthe computers may be used.

An auxiliary subsystem 799 (e.g., for auxiliary display of content) maybe connected via the user interface 760 to allow data such as programcontent, system status and event notifications to be provided to theuser, even if the main portions of the computer system are in a lowpower state. The auxiliary subsystem 799 may be connected to the modem772 and/or network interface 770 to allow communication between thesesystems while the main processing unit 720 is in a low power state.

CONCLUSION

While the invention is susceptible to various modifications andalternative constructions, certain illustrated embodiments thereof areshown in the drawings and have been described above in detail. It shouldbe understood, however, that there is no intention to limit theinvention to the specific forms disclosed, but on the contrary, theintention is to cover all modifications, alternative constructions, andequivalents falling within the spirit and scope of the invention.

1. A method for recognizing handwritten input data, the methodcomprising combining, by a computer in response to recognizing thehandwritten input data, a first set of scores provided by an offlinerecognizer with a second set of scores provided by an online recognizerin response to recognizing the handwritten input data, the combiningbased on a repeated base learning algorithm.
 2. The method of claim 1wherein the base learning algorithm produces a combination function in afeature space composed of the first set of scores and the second set ofscores.
 3. The method of claim 2 wherein the combining is performed bythe produced combination function.
 4. The method of claim 2 wherein thefirst set of scores and the second set of scores comprise a featurepool.
 5. The method of claim 4 further comprising applying an AdaBoostfunction to the feature pool.
 6. The method of claim 1 furthercomprising returning, in response to the combining, a single result thatrepresents the recognized handwritten input data.
 7. The method of claim1 further comprising returning, in response to the combining, a resultset with each result in the set scored, each scored result representingthe recognized handwritten input data.
 8. At least one computer storagemedia storing computer-executable instructions that, when executed by acomputer, cause the computer to perform a method for recognizinghandwritten input data, the method comprising combining, in response torecognizing the handwritten input data, a first set of scores providedby an offline recognizer with a second set of scores provided by anonline recognizer in response to recognizing the handwritten input data,the combining based on a repeated base learning algorithm.
 9. The atleast one computer storage media of claim 8 wherein the base learningalgorithm produces a combination function in a feature space composed ofthe first set of scores and the second set of scores.
 10. The at leastone computer storage media of claim 9 wherein the combining is performedby the produced combination function.
 11. The at least one computerstorage media of claim 9 wherein the first set of scores and the secondset of scores comprise a feature pool.
 12. The at least one computerstorage media of claim 11 further comprising applying an AdaBoostfunction to the feature pool.
 13. The at least one computer storagemedia of claim 8 further comprising returning, in response to thecombining, a single result that represents the recognized handwritteninput data.
 14. The at least one computer storage media of claim 8further comprising returning, in response to the combining, a result setwith each result in the set scored, each scored result representing therecognized handwritten input data.
 15. A system configured forrecognizing handwritten input data, the system comprising: a computer, acombiner mechanism implemented at least in part by the computer andconfigured for combining, in response to recognizing the handwritteninput data, a first set of scores provided by an offline recognizer witha second set of scores provided by an online recognizer in response torecognizing the handwritten input data, the combining based on arepeated base learning algorithm.
 16. The system of claim 15 wherein thebase learning algorithm is configured to produce a combination functionin a feature space composed of the first set of scores and the secondset of scores.
 17. The system of claim 16 wherein the combining isperformed by the produced combination function.
 18. The system of claim16 wherein the first set of scores and the second set of scores comprisea feature pool.
 19. The system of claim 18 further comprising thecomputer configured for applying an AdaBoost function to the featurepool.
 20. The system of claim 15 further comprising the computer furtherconfigured for returning, in response to the combining, a result thatrepresents the recognized handwritten input data.